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Abstract 



Given a discrete-time linear system C, a shortest basis for C is a set of linearly independent 
generators for C with the least possible lengths. A basis B is a shortest basis if and only if it has 
the predictable span property {i.e., has the predictable delay and degree properties, and is non- 
catastrophic), or alternatively if and only if it has the subsystem basis property (for any interval 
J7\ the generators in B whose span is in J is a basis for the subsystem Cj). The dimensions of 
the minimal state spaces and minimal transition spaces of C are simply the numbers of generators 
in a shortest basis B that are active at any given state or symbol time, respectively. A minimal 
linear realization for C in controller canonical form follows directly from a shortest basis for C, 
and a minimal linear realization for C in observer canonical form follows directly from a shortest 
basis for the orthogonal system C . This approach seems conceptually simpler than that of 
classical minimal realization theory. 

Keywords: linear systems, minimal realizations 



I. Introduction 

The minimal realization problem of linear system theory is the problem of finding a state-space 
realization for a given linear system, often time- invariant, that has the smallest possible state 
space(s), possibly in some predetermined canonical form. The problem becomes nontrivial in the 
general case of multivariable and/or time- varying linear systems. The system is usually specified 
by its impulse response(s), or by some realization that may be nonminimal. 

This problem has been studied since the rise of the state-space paradigm in the early 1960s. 
The classical solution to this problem is usually expressed by the mantra "minimal = controllable 
+ observable." Many concrete algorithms have been developed to solve it, typically making heavy 
use of linear algebra and matrix manipulations; see e.g., [TJ. 

There is a much simpler approach to the minimal realization problem, at least conceptually; 
namely, the "shortest basis" approach, as we shall call it here. For linear time-invariant systems, 
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1 \n other words, everything should be made as simple as possible, but not simpler. 



It can scarcely be denied that the supreme goal of all theory is to make 
irreducible basic elements as simple and as few as possible. — A. Einstein 
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this approach was developed in [6]. It has been used extensively in the literature of minimal trellis 
(state-space) realizations of linear block codes [121 [H], where shortest bases are called "trellis- 
oriented" [7] or "minimum-span" generator matrices. Analogous results are developed in a very 
general group-theoretic setting in [8]. This paper may be regarded either as a specialization of [8] 
to the case of linear systems over fields, or, preferably (because all proofs are linear-algebraic), a 
generalization of [6] to time-varying linear systems, or of [12] to infinite-time-axis linear systems. 

Much of this paper is a tutorial overview of known results, aimed particularly at a system-theory 
audience. The following two results are new, as far as we know: 

• B is a shortest basis if and only if it has the predictable span property; 

• B is a shortest basis if and only if it has the subsystem basis property. 

But many of these results are not very well known, at least in system theory, and many of the 
proofs are new. 

II. Preliminaries 

We focus on linear discrete-time systems over a field F. In system theory, F is usually the real field 
R or the complex field C, whereas in coding theory F is usually a finite field. The astute reader 
will notice that most proofs depend only on the group property of linear systems, and therefore 
apply more generally to group systems. The reader will also observe that our approach is entirely 
algebraic, and that analytical issues such as stability and convergence play no role. 

A discrete-time system has a discrete, ordered time axis X, which we take to be the set of 
integers Z, or a subinterval of Z. We use notation such as [n, m) = {k € Z : n < k < m} for 
subintervals of Z. 

Here, as in behavioral system theory [15] . a system will be defined by the set C of all of its 
possible trajectories a = {a/% : k € 1} (its "behavior"), where each symbol a\. lies in some alphabet 
Ak- If the system is linear over a field F, then each alphabet Aj~ is a vector space over F, assumed 
to be finite-dimensional, and C is a subspace of the Cartesian-product vector space A = Wk&x^k- 

If all alphabets are equal to F, then a trajectory a may be represented by its z-transform 
a{z) = akZ~ k , as in linear system theory, or by its D-transform a(D) = a,kD k , as in coding 
theory. The subtle difference (apart from the obvious difference D = .z" 1 ) is that D is simply 
an indeterminate, whereas z is often regarded as a complex variable. We shall use both kinds of 
transforms in this paper, but we shall always regard z as simply an indeterminate. 

The support of a trajectory a is the subset of all indices k € I such that a& ^ 0. A trajectory 
is zero, finite or infinite according to whether the size of its support is zero, finite or infinite. 

If the support of a nonzero trajectory a has a minimum element fc m i n , then a is called Laurent, 
and k m i n is called the delay of a, denoted by del a = k m \ n . By convention, the zero trajectory is 
defined as Laurent, and its delay is defined as del = +oo. 

In the body of this paper, we shall require all trajectories to be Laurent, as is common in coding 
theory. This restriction simplifies our exposition and yields a symmetrical duality theory, but fore- 
closes consideration of uncontrollable systems with autonomous components. In an appendix, we 
show that the extension of our approach to uncontrollable/autonomous systems is straightforward. 

If the support of a has a maximum element fe max , then fe max is called the degree of a, denoted 
by dega = k max . By convention, the degree of the zero trajectory is defined as degO = — oo. 
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The set of all z-transforms a(z) or ^-transforms a{D) of Laurent trajectories a over F on the 
time axis X = Z is called the set of all formal Laurent series in F over z~ l or D, and is conventionally 
denoted by F((z -1 )) or ¥((D)), respectively. A nice algebraic property of F((z -1 )) or ¥((D)) is that 
it forms a field, with multiplication defined by sequence convolution. In particular, every nonzero 
a(z) E F((z~ 1 )) has a Laurent inverse l/a(z), which may be computed by long division. 

The set of all z-transforms a(z) or D-transforms a(D) of finite trajectories a over F with 
del a > is called the set of all polynomials in F over or D, and is denoted by F[z _1 ] or F[Z?], 
respectively. For polynomials, our definition of "degree" coincides with the standard definition. 

A linear time-invariant (LTI) system is a linear system C whose time axis is I = Z, whose 
alphabets Ak are all equal, and which satisfies DC = C, where D is the delay operator that trans- 
forms a € C to Da = {at-i ■ k G I}. (This usage of D is compatible with that in D-transforms, 
since if the L>-transform a is a(D), then that of Da is Da(D).) This implies that if a £ C, then 
every positive or negative shift D k a, k E Z, is in C. Note that the set ¥((D)) of all formal Laurent 
series over F is time-invariant. 

It is natural to define a linear system C by a linearly independent set g of generators, called 
a basis, such that every trajectory in C is a unique linear (over F) combination of the generators. 
We say that a generator is involved in a linear combination if it has a nonzero coefficient in that 
combination. For an LTI system, it is natural to choose a basis that consists of all the shifts 
D k gj, k £ Z, of a set {gj} of fundamental generators gj. 

Example 1 (single-input, single-output LTI system). Consider a real or complex discrete-time 
linear filter whose impulse response has z-transform g(z) = 1/(1— /3z~ l ), which denotes the Laurent 
z-transform 1 + f3z~ l + (3 2 z~ 2 + • • - H What is the set C of trajectories associated with this filter? 
We might say that C is the set of all output sequences of the filter in response to all Laurent 
input sequences. But then C would simply be the set of all Laurent sequences, since every Laurent 
sequence a(z) could be the output sequence if the input were the Laurent sequence a(z)(l — /3z~ l ); 
so such a definition would fail to capture the particular characteristics of this filter. Therefore we 
instead define the set C of trajectories of this system as the set of all input-output pairs as the 
input runs through all Laurent sequences: 

C = {(u(z),u(z)g(z)) : u(z) eFdz- 1 ))}. 

C is evidently an LTI system. The set of all shifts of the fundamental input-output pair (l,g(z)) is 
a basis for C. □ 

Example 2 (binary linear block code). The (8,4,4) first-order binary Reed-Muller code is the 
four-dimensional subspace C of (F2) 8 that comprises all 16 binary 8-tuples that can be obtained as 
binary linear combinations of the following four generators: 

gi = 11110000; 
g 2 = 11001100; 
g 3 = 10101010; 
g 4 = 11111111. 

C may be regarded as a linear system over the binary field F2 that is defined on the finite time axis 
X = [0, 8), with the basis given above. Of course, a system defined on a finite time axis cannot be 
time-invariant. □ 

2 We need not restrict < 1 if we are not concerned with issues of stability or convergence. 
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Given a linear system C C A = Ukex^k defined on a time axis I, a (state-space) realization of 
C (sometimes called a "trellis realization" in coding theory) will be defined in terms of: 

• A state time axis Is Q Z, such that symbol time k E I occurs between state time k Els an d 
state time k + 1 E Is- If X = Z, we take Z5 = Z; but if I = [0, re), we take Is = [0, re]. 

• A set of state spaces £ X5. 

• For each k El, a set 7fc of allowable transitions (cr^, a^, Cfc+i) £ x ij. x S/t+i- 

The /reZZ behavior 23 of the realization is then the set of all symbol-state trajectories (a, <x) € 
„4 x IIfc g x s Sfc such that (ofc, a^, Cfc+i) E Tk for all k El. The system C realized by the realization 
is the set of all symbol trajectories a E A that appear in some symbol-state trajectory (a, a) E 03. 

A realization is linear if all symbol alphabets and state spaces are vector spaces over a field 
F, and the transition spaces Tk ( "local behaviors" ) are subspaces of the vector spaces S^xAjX £fc+i 
for all k El. A realization is minimal if the state spaces are as small as possible for all k Els- 
We will see that every linear system C has a realization that is both linear and minimal. 

III. Minimal state and transition spaces 

In this section we recapitulate well-known results about minimal state spaces, and less well-known 
results about minimal transition spaces. 

A. Minimal state spaces 

A fundamental result of Willems' behavioral system theory |15[ [To] is that, given a linear system 
C, the minimal state space at each possible state time is unambiguously defined. 

A state space at state time k Els may be considered to be defined by a cut of the symbol 
time axis I between symbol time k — 1 and symbol time k. Such a cut partitions I into two disjoint 
subintervals: a past k~ = {k 1 E I : k! < k} and a future k + = {k' E I : k! > k}. 

The fundamental property of states is the Markov property: the future should be conditionally 
independent of the past, given the state. In a state-space realization, this translates to a requirement 
that two symbol trajectories up to time k — 1 may arrive at the same state in if and only if the 
sets of their possible future continuations from time k on are identical]! 

In the linear case, it is easy to identify when this happens. Let the past subsystem C^- and 
future subsystem C k + be defined as the subsets of C that are all-zero on the future k + and the past 
k~ , respectively. Both and are evidently linear subsystems of C. Then the minimal state 
space at state time k is the following quotient space: 



Ck- x C/c+ 

The proof is essentially as follows. The quotient space C/(Cf.- x C^+) is a disjoint union of cosets 
a + (Cf.- x Cfc+), where each coset representative a is a trajectory in C. Define P^- : A — > A and 
: A — > A as the projection operators onto the past k~ and future k + , respectively. The coset 
a + (Cfc- x Cfc+) is then precisely the Cartesian product 

3 In automata theory, this is called Nerode equivalence. 
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a + (C fc - x C fe + ) = (P k - (a) + C k - ) x (a) + C fc+ ) ; 



i.e., the set of trajectories in C whose past projections are in the coset P k -(a) + C k - of C k ~, and 
whose future projections are in the coset P k +(a) + C k + of C k +. Since every such past projection has 
the same set of future continuations, all of these trajectories may pass through the same state at 
time k. On the other hand, any past projection in any other coset may not pass through the same 
state at time k, since it has a disjoint set of future continuations. Thus any minimal state space 
must be in one-to-one correspondence with this set of cosets; i.e., with T, k . 

By a simple extension of the above argument, or by an elementary result in group theory (the 
first theorem about subdirect products in [TO]), the minimal state space T, k is also isomorphic to 
the following quotient spaces, called the "past-induced" and "future-induced" state spaces: 

where P k -{C) and P k +(C) are the sets of past and future projections of C, respectively. 

It is straightforward to define a linear state-space realization of a linear system C that uses the 
minimal state spaces for every time k. Let each trajectory a G C pass through the sequence 
of states <7fc(a) G T, k for all k that are defined by the natural maps from C to the quotient spaces 
C/(C k ~ x C k +). This is called the canonical realization of a linear system C. Among other things, 
the canonical realization shows that there exist linear realizations whose state spaces are minimal 
at every time k G Is- 

We shall require that all minimal state spaces T, k be finite-dimensional. As we shall see shortly, 
this condition ensures that only finitely many generators in a shortest basis B will be "active" at 
any state time k, which in turn ensures that the number of generators that affect any component 
a k of any trajectory a G C will be finite, even when B is infinite. 



B. Minimal transition spaces 

We now discuss minimal transition spaces. In coding theory, transition spaces ("branch spaces," 
"trellis sections," "local constraint codes") have come to be regarded as having importance equal 
to or possibly even greater than that of state spaces. 

Minimal transition spaces are characterized by the following theorem: 

Theorem 1 (Minimal transition spaces) In any minimal realization of a linear system C, for 
every symbol time k G X, the set of transitions is in one-to-one correspondence with the following 
quotient space, called the minimal transition space at symbol time k: 



Ck- x C(k+\)+ 

Proof. In a minimal realization, the state spaces T, k and 'Sk+i are in one-to-one correspondence 
to P k -/C k - and P(k+i)+/C(k+i)+> respectively. The set of all trajectories a G C that pass through 
a given transition (a k , a k , cr k +i) G T, k x A k x is the set that have a past projection P fc -(a) 

in the coset of C k - that corresponds to a k , a time-fc projection P{ fc }(a) equal to a k , and a future 
projection P( fc+1 )+(a) in the coset of C( fc+1 )+ that corresponds to a k+ i. Thus the trajectories of C 
that pass through the same transition at symbol time k are precisely those that lie in the same 
coset of C k - x C( fc+1 )+, so the set of transitions corresponds one-to-one to the set T k of cosets of 
Ck- x in C. □ 
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It is easy to see that in a canonical realization, each trajectory a € C passes through the sequence 
of transitions that are defined by the natural maps from C to the quotient spaces C/{C k - x C k+1 +), 
where <7fc(a) and <Tfc+i(a) are identified with past-induced and future-induced states, respectively. 

IV. Shortest bases and minimal realizations 

It is convenient to have a basis for a linear system C from which a minimal state-space realization of 
C and its parameters can be read directly, "by inspection." In this section we will see that a shortest 
basis has these properties. In the literature of minimal trellis realizations of linear block codes, 
such a shortest basis is called a "trellis-oriented" or "minimum-span" generator matrix [12\ [T4"] . In 
system theory terms, it yields a minimal realization of C in "controller canonical form" 

We first mention some technical "well-behavedness" requirements that we impose on C for 
simplicity. Let Cfinite denote the set of all finite trajectories in C. We say that a linear combination 
of trajectories is Laurent if it involves only trajectories whose delays are not less than some minimum 
delay /c m i n ; then the combination is a Laurent trajectory with delay > k m \ n . We require that C be 
the set of all Laurent linear combinations of Cfinite- Then C is generated by its finite trajectories; 
such a linear system is called controllable \16\ [3], Also, C is then complete [16] in a Laurent sense; 
indeed, C is the Laurent completion of C nn it e . For an extension of our development to uncontrollable 
systems, see the Appendix, or [8]. 

A. Shortest bases 

The span of a nonzero finite trajectory a G A is the shortest interval that contains its support, 
namely [del a, dega], and the length of a is the size of its span, namely dega — del a + 1. 

Loosely, a shortest basis B for a linear system C will be defined as a basis whose elements are 
as short as possible. We will see as we proceed that this concept is well defined. 

Example 1 (cont.). Recall that the single-input, single-output linear time-invariant system of 
Example 1 is defined as the set C of all input-output pairs 

C = {u(z)(l,g(z)):u(z)e¥((z- 1 ))} 

as the input u(z) runs through all Laurent sequences, where g(z) = 1/(1 + fiz~ v ). In other words, 
C is the set of all multiples of the basic input-output pair (1, g(z)) by sequences in the Laurent field 
F((z -1 )). It is easy to see that a nonzero trajectory in C is finite if and only if the corresponding 
input sequence u(z) is a finite multiple of l/g(z) = l+fiz" 1 . Hence the set C nn ite of finite trajectories 
in C is the set of multiples of the finite pair (1 + fiz~ l , 1) by finite Laurent sequences v(z): 

Cfinite = {(v(z){l + f3z- l ),v(z)) : v(z) € (F((z- 1 ))) finite }, 

From this it is clear that the shortest trajectories in C have length 2, and that the set of shifts of 
the length-2 trajectory (1 + fiz^ 1 , 1) is a shortest basis B for C. □ 

Example 2 (cont.). Recall that the (8,4,4) binary Reed-Muller code of Example 2 is the set C of 
all 16 binary linear combinations of the four generators (11110000, 11001100, 10101010, 11111111). 
By examining all 16 codewords, we find that the shortest nonzero codewords are the three length-4 
8-tuples 11110000,00111100 and 00001111, which are evidently independent. The next-shortest 
codeword that is independent of the previous three is the length-6 8-tuple 01011010. Since the 
dimension of C is 4, these four 8-tuples comprise a shortest basis B for C. □ 
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A shortest basis B may be found by the following greedy construction. Recall that if J is a 
subinterval of the time axis Z, then Cj denotes the subsystem of C consisting of all trajectories 
a € C whose span is contained in J . First, for every length-1 subinterval J C Z, find a set of 
linearly independent length-1 generators for Cj. Next, for every length-2 subinterval J C Z, find 
a minimal set of additional independent length-2 generators sufficient (in combination with the 
previous length-1 generators) to generate all length-2 trajectories in Cj. And so forth. 

Assuming that C is controllable, this algorithm will eventually find a shortest basis B of finite 
linearly independent generators for C. Furthermore, it is clear that any shortest basis for C may be 
constructed in this way. Since dim Cj is a parameter of the system C for every subinterval J C Z, 
it follows that the set of lengths of generators in any shortest basis B for C is the same. 

B. The predictable span property and the subsystem basis property 

We now introduce two properties, the predictable span property and the subsystem basis property, 
and show that a basis B is a shortest basis if and only if it has either of these properties. 

We note that linear independence implies that every trajectory a € C has a unique expression as 
a (possibly infinite) linear combination of generators in B, so that we may speak of the generators of 
a, meaning the subset 5(a) of generators of B that are involved in this unique linear combination. 

If 5(a) is the set of generators of a trajectory a £ C, then del a > k m \ n = min ggi 5( a ) del g, and 
dega < £; m ax = max ggiS ( a ) deg g, where strict inequality may occur due to cancellations. A basis 
B has the predictable span property (PSP) if inequality never occurs; i.e., if the span of a is 
always equal to [k^, k mBX ]. 

For finite linear combinations, B evidently has the PSP if and only if it has the predictable 
delay property (i.e., del a = k m \ n always) and the predictable degree property (i.e., dega = fe max 
always) [6]. Clearly B has the predictable delay property if and only if the time-/c symbols of the 
delay-/c generators in B are linearly independent, so cancellation can never occur; similarly B has 
the predictable degree property if and only if the time-/c symbols of the degree-/c generators in B 
are linearly independent. 

These linear independence properties have an immediate corollary: 

Lemma 2 (A finite number of generators start and stop at each time) // a basis B for C 
has the predictable delay (resp. degree) property, then the number of generators in B that have 
delay k (resp. degree k) is not greater than dimA^. In particular, if dim A/% = 1, then at most one 
generator in B can start or stop at time k. 

Proof. If B has the predictable delay property, then the set of time-/c symbols of delay-/c generators 
in B is a linearly independent subset of elements of the time- A: symbol alphabet A&; similarly the 
set of time-fc symbols of degree-A; generators in B is a linearly independent subset of A&. □ 

Now let us consider infinite Laurent linear combinations. Recall that a linear combination 
of generators is Laurent if it involves only generators whose delays are not less than some finite 
minimum delay k m i n . On the other hand, we must have fc max = since an infinite number of 
generators are involved and there can only be a finite number of each finite degree, under our 
assumption that dimA^ is finite. Therefore B has the PSP for infinite linear combinations if and 
only if it has the predictable delay property and all infinite linear combinations are infinite. (See 
Section V for examples of infinite linear combinations that are finite.) 
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Borrowing a term from coding theory, we say that a basis B for C is catastrophic if there exists 
a finite a G C that is equal to an infinite linear combination of generators. Thus, in summary, B has 
the PSP if and only if B has the predictable delay and degree properties, and B is non-catastrophic. 

Secondly, we will say that a basis B has the subsystem basis property (SBP) if for any 
subinterval J of the time axis I, the set of generators in B whose span is contained in J is a basis 
for the subsystem Cj. By construction, a shortest basis has this property for all finite J . 

Now we show that a basis B is a shortest basis if and only if it has the PSP, or the SBP: 

Theorem 3 (Shortest basis 44> PSP ^ SBP) For a basis B of a controllable linear system C, 
the following are equivalent: 

1. B has the predictable span property; 

2. B has the subsystem basis property; 

3. B is a shortest basis for C . 

Proof. (1 =4> 2) On the one hand, a linear combination of generators in B whose span is contained 
in J must be a trajectory a G Cj. On the other hand, if B has the PSP and a G Cj, then the 
minimum degree of the generators of a is del a G J : and the maximum degree is dega G J, so 
every a G Cj is a linear combination of generators in B whose span is contained in J. 

(2 => 3) If B has the SBP, then, for each finite subinterval J, the generators in B whose span is 
precisely J could be chosen in the shortest-basis construction process, so B is a shortest basis. 

(3 1) If B is a shortest basis for C, then the set of time-A symbols of delay-A generators must be 
a linearly independent subset of the time-A; symbol alphabet A k , else there would be a finite linear 
combination b of delay-A generators with del b > k, and with degb < A max = degg, the degree 
of a greatest-degree generator g involved in this linear combination, so b would be shorter than 
g, and b could replace g in B to produce a shorter basis B'; contradiction. So B must have the 
predictable degree property. Similarly, B must have the predictable delay property. Finally, by the 
shortest-basis construction, every finite trajectory a G C is uniquely expressible as a finite linear 
combination of finite generators in B, so B must be non-catastrophic. □ 

C. Dimensions of minimal state and transition spaces 

We now show how to determine the dimension of the minimal state space T, k = C/(C k - x C k +) "by 
inspection" from any shortest basis B for a linear system C, for any cut of the time axis I into a 
past subset k~ and a future subset k + . 

We partition the generators in B into three subsets: a past subset S k - consisting of those 
generators in B whose support is contained in A - , a future subset S k + consisting of those generators 
in B whose support is contained in k + , and a remainder subset R k consisting of the remaining 
generators, which we call the active generators at state time k G Is- By the subsystem basis 
property of shortest bases, S k - is a basis for C k - and S k + is a basis for C k +. 

Theorem 4 (Minimal state space dimension) For any state time k G Is, the dimension of 
the minimal state space T, k of a linear system C is the number of active generators at state time k 
in any shortest basis B for C. 
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Proof. Since B is a basis for C, S k - is a basis for C k -, and S k + is a basis for C k +, the quotient space 
Sfc = C/{C k - x C fc +) has a basis consisting of the remaining linearly independent generators in the 
remainder subset R k , and thus has dimension \R k \. □ 

Example 1 (cont.). Recall that a shortest basis for the system C of Example 1 is the set of shifts 
of the length-2 trajectory (1 + fiz" 1 , 1). For any state time k G Z, precisely one of these generators 
is active; for example, at state time 1, only the fundamental generator (1 + /3z _1 , 1) is active, since 
all of its shifts have supports either in the past l~ = {/e<l}orin the future 1 + = {k > 1}. Thus 
the dimension of the minimal state space T, k of C at any state time k G Z is 1. 

More generally, let C be any single-input, single-output LTI system consisting of all input-output 
trajectories {u(z)(l, g(z)) : u(z) G F((z -1 ))}, where g(z) is a causal rational impulse response 
g{z) = a(z)/b(z) with a(z),b(z) being relatively prime polynomials in F[z _1 ], with del a(z) > 
and del b{z) = 0. Then a shortest basis for C is the set of all shifts of the fundamental input-output 
trajectory (b(z),a(z)), whose delay is and whose degree is 5 = maxjdeg b(z), deg a(z)}. Precisely 
5 of these shifts are active at any state time k G Z; therefore the dimension of the minimal state 
space Sfc of C at any state time k G Z is 5. □ 

Example 2 (cont.). Recall that the (8,4,4) code of Example 2 has the following shortest basis: 

gi = 11110000; 
g 2 = 00111100; 
g 3 = 00001111; 
g 4 = 01011010. 

Notice that the generators in this set "start" at symbol times 0, 1, 2 and 4, and "stop" at symbol 
times 3, 5, 6 and 7. The numbers of generators that are active at state times 0, 1, . . . , 8 are therefore 
0,1,2,3,2,3,2,1,0, respectively, which are therefore the minimal state space dimensions at these 
times. We say that the state-space dimension profile of C is {0, 1, 2, 3, 2, 3, 2, 1, 0}. □ 

Similarly, from the minimal transition space theorem, we can determine the dimension of the 
minimal transition space Tk from any shortest basis B for a linear system C. We now partition the 
time axis X into three subintervals: a past interval k~ , the time {k}, and a future interval (k + 1) + . 
We partition the generators in B into three subsets: a past subset S k ~ consisting of those generators 
in B whose support is contained in A; - , a future subset consisting of those generators in 

B whose support is contained in (k + 1) + , and a remainder subset T k consisting of the remaining 
generators, which we call the active generators at symbol time k el. 

Theorem 5 (Minimal transition space dimension) For any symbol time k G 1, the dimen- 
sion of the minimal transition space T k of a linear system C is the number of active generators at 
symbol time k in any shortest basis B for C. 

Proof. Since B is a basis for C, S k - is a basis for C k -, and Sr k+1 \+ is a basis for Co,+i)+, the 
quotient space Tk = C/(C k - x C( fe+1 )+) has a basis consisting of the remaining linearly independent 
generators in the remainder subset T k , and thus has dimension \T k \. □ 

Example 1 (cont.). For the system C of Example 1, we may take B as the set of shifts of the 
length-2 trajectory (1 + /3z _1 , 1). For any symbol time fceZ, precisely two of these generators are 
active; for example, at symbol time 1, the generators (1 + fiz -1 , 1) and (z^ 1 + fiz~ 2 , z^ 1 ) are both 
active. Thus the dimension of the minimal transition space Tk of C at any symbol time k G Z is 2. 
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More generally, let C be any single-input, single-output LTI system whose shortest-generator 
set is the set of all shifts of the fundamental input-output trajectory (b(z),a(z)), whose delay is 
and whose degree is 5 = maxjdeg b(z), deg a(z)}; then precisely 5 + 1 of these shifts are active at 
any symbol time k £ Z, so the dimension of the minimal transition space Tk of C at any symbol 
time k € Z is <5 + 1. □ 

Example 2 (cont.). Given the shortest basis {11110000,00111100,00001111,01011010} for the 
binary code C of Example 2, we observe that the numbers of generators that are active at symbol 
times 0,1,... ,7 are 1,2,3,3,3,3,2,1, respectively. Thus these are the minimal transition space 
dimensions. We say that the transition- space dimension profile of C is {1, 2, 3, 3, 3, 3, 2, 1}. □ 

In coding theory, the transition-space dimension profile of a linear code C is generally taken as 
a better measure of the complexity of trellis-based decoding than its state-space dimension profile. 

D. Minimal realizations in controller canonical form 

Given any shortest basis B for any linear system C, we can now construct an obvious state-space 
realization for C, sometimes called the controller canonical form [11], which is evidently minimal. 
For multivariable LTI systems, a construction of a minimal realization in controller canonical form 
from a shortest ("minimal") basis was given in [UJ. In the literature of trellis realizations of block 
codes, such a construction was first given by Kschischang and Sorokine [12], who introduced the 
term "atomic." 

With each generator g € B, we associate an atomic state-space realization as follows. Roughly, 
it involves an "input" a € F that occurs at symbol time del g; a "memory element" that stores 
a during the active state interval (del g, degg]; and an "output" whose value during the active 
symbol interval [del g, deg g] is ag. 

More precisely, if del g < deg g, then the state spaces of the atomic realization are equal to F 
during the active state interval (del g, degg], and equal to the trivial space {0} otherwise^ thus 
the state space dimension is 1 during the active interval and otherwise. The sets of allowable 
transitions Tk are as given below during the active symbol interval [del g, deg g] (otherwise Tk = 
{(0,0,0)}): 

• For k = del g, Tk = {(0, ag k ,a) : a £ F}; 

• For del g < k < degg, Tk = {(a,ag k ,a) : a € F}; 

• For k = degg, T k = {(a,ag k ,0) : a G F}; 

thus the transition space dimension is 1 during the active interval, and otherwise. 

The full behavior of this atomic realization is thus the one-dimensional vector space 23 = {(a = 
ag, <r = al(dei g ,degg]) : a G F}, where l^ei g.degg] is the indicator function of the state interval 
(del g, degg]. The system that it realizes is the one-dimensional vector space {ag : a € F}; i.e., 
the subsystem of C that is generated by g. 

4 If del g = degg, then = {0} for all k £ Xs, and Tk = {(0, agk, 0) : a £ F} at symbol time k = del g = degg; 
otherwise Tk = {(0,0,0)}. 
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The whole state-space realization for C then consists of the aggregate of these atomic realizations 
for all g € £>, plus an adder which at each symbol time produces the sum of the outputs of 
the currently active atomic realizations. The set of all possible output trajectories of the whole 
realization is thus the set of all linear combinations X^geB a (§)§ °f generators in B, which is precisely 
the linear system C. The number of memory elements active at any state time k € Is is the number 
of active generators at time k, which by the theorem above is the dimension of the minimal state 
space Sfc for C. Thus this aggregate "controller canonical form" realization is a minimal (and linear) 
realization of C. 

In a controller canonical realization of a linear time-invariant system, the lengths of the gen- 
erators g G B are sometimes called the controllability indices of C. Thus in a linear time- varying 
system, the lengths of the generators may be regarded as generalized controllability indices. 

E. New information, and forgetting information 

Further important quantities in a linear system C are the amount of information that enters or 
"drives" the system at each time, and also the amount that is "forgotten" at each time. 

We define the in-space Ik at symbol time k £ I as the quotient space C k + /C( k +i)+'> i-e., the set 
of trajectories in C that start at time k or later, modulo those that start at time k + 1 or later. 

If B is a shortest basis for C, then is generated by the elements of B that have delay k or 
more, and C(u + \\+ is generated by the elements of B that have delay k + 1 or more; therefore: 

Theorem 6 (In-space dimension) For any symbol time k € I, the dimension of the in-space 
of a linear system C is the number of delay-k generators in any shortest basis B for C. 

As we have seen previously, the time-A; symbols of delay-/c generators in a shortest basis B must 
be linearly independent, so their number must satisfy < dirnJ^. < dimAk- 

Now if we compare the minimal state space T, k = C/(C^- x C k +) to the minimal transition space 
7fc = C/(C k - x C( fc+1 )+), we see that 

dim7fc = dimSfc + dimifc. 

In words, if we regard one element of the field F as one unit of information, then in a minimal 
realization of C the transition at symbol time k is completely determined by dim T, k units of state 
information at state time k, plus dimT^ new units of information at symbol time k. Indeed, in 
the controller canonical realization, we see explicitly that the system transition at symbol time k 
is completely determined by the coefficients of the dimS^, generators that are active at state time 
k, plus the dim/fc generators that start at symbol time k. 

As an obvious corollary, if the minimal state spaces and the symbol alphabets A k are finite- 
dimensional, then the minimal transition spaces Tk are finite-dimensional. 

Symmetrically, we may define the out-space Ok at symbol time G I as the quotient space 
C(k + i)-/Cfc- ; then the dimension of Ok is the number of degree-/c generators in any shortest basis 
B for C, and we have dim7fc = dimSfc + i + dimO^, where < dimO^ < dimAk- In words, we may 
interpret dimO^ as the number of units of information that are "forgotten" at symbol time k, as 
may be seen explicitly in the controller canonical realizationj^] 

5 Alternatively, if we were to run the system in the reverse-time direction, then dimO^ and dim 7^ would reverse 
roles and become the amounts of "new" and "forgotten" information, respectively. 
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V. Minimal realizations of multivariable LTI systems 



We now consider minimal realizations of general Laurent LTI systems, which for brevity we simply 
call multivariable LTI systems. The material in this section is well known; our purpose is to let the 
reader see its connections with the "shortest basis" approach. 

As usual, we make no distinction between (F((z _1 ))) n and F n ((z -1 )). 

A multivariable LTI system is usually defined (as in Example 1) by an n x k rational generator 
matrix G{z) = {gij(z) : 1 < i < n, 1 < j < k}, where each element g%j{z) is a rational Laurent 
sequence. A Laurent sequence in F((z -1 )) is rational if it can be expressed in the form a(z)/b(z), 
where a(z) and b(z) are polynomial sequences in Flz- 1 ] with b(z) ^ 0. The set of all rational 
Laurent sequences is denoted by ¥(z~ 1 ), and forms a field; in particular, the multiplicative inverse 
of a nonzero rational sequence a(z)/b(z) is b(z)/a(z). 

The system C is then the set of all n-tuples of Laurent sequences y(z) = G(z)u(z) as u(z) 
ranges through all fc-tuples of Laurent sequences: 

C = {y(z)=G(z)u(z):u(z)e(¥((z- 1 ))) k }. 

(Notice that the component sequences yi(z) of y(z) may represent either "inputs" or "outputs," as 
for example in Example 1.) It follows that such an LTI system C is a subspace of the n-dimensional 
vector space (F((z _1 ))) n of all n-tuples of Laurent sequences over the Laurent field F((z -1 )). Thus 
without loss of generality we may assume that k < n, and that G{z) has full rank k. 

We now give an "algorithm" to find a shortest basis B for C. The correctness of this approach 
is proved in |4j, using the invariant factor theorem (IFT), and again in [6], without using the IFT. 
The "shortest basis" approach gives a nice motivation for this development. 

We first find a set of k independent finite generators whose shifts generate C, by finding the 
shortest finite trajectories in the k 1-dimensional subsystems Cj that are generated by the k ratio- 
nal generators gj(z) = {gij(z) = aij(z)/b{j(z) : 1 < i < n}. We observe (as in Example 1) that 
Uj (z)gj (z) is finite if and only if Uj (z) is a finite multiple of Xj (z) / 7^ (z) , where Xj (z) is the least com- 
mon multiple of the denominators bij(z), and ^j{z) is the greatest common divisor of the numerators 
aij(z). The shortest finite trajectories in Cj are therefore the shifts of g'j(z) = Xj(z)gj(z)/~/j(z). 
Since we may take any shift of g'j(z), we may assume without loss of generality that the delay of 
g'j(z) is zero. Then the polynomial matrix G'(z) = {g'j(z) : 1 < j < k} is a basis for C. 

We note in passing that this construction shows that any LTI system C that is generated by a 
rational generator matrix G{z) is controllable; i.e., generated by its finite trajectories. 

Now we ask whether there exists any finite linear combination G'(z)u(z) of the polynomial 
generators g'j(z) that produces a finite sequence that is shorter than any generator involved in this 
combination. As we saw earlier, this can happen if and only if the set of shifts of the {g'j(z)} does 
not have the predictable delay property or the predictable degree property. 

1. The set of shifts of the {g'j(z)} does not have the predictable delay property if and only if 
there exists a linear combination g"{z) of the delay-0 generators g'j(z) that has delay greater 
than zero. This occurs if and only if the delay-0 coefficient n-tuples g'j are linearly dependent 
over F, which occurs if and only if the k x k minors (determinants of k x k submatrices) of 
the n x k matrix G'(z) all have delay greater than zero. In this case we can obtain a shorter 
set of generators by substituting the linear combination g"(z) for a longest generator g'j(z) 
that is involved in the combination. 
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2. The set of shifts of the {g^(z)} does not have the predictable degree property if and only if 

there exists alinear combination g"(#) of the degree-0 shifts D~ dess ^ z ^g'j(z) of the generators 
gj(z) that has degree less than zero. This occurs if and only if the high-order coefficient n- 
tuples gjdegg'. are linearly dependent over F, which occurs if and only if the k x k minors of 

G'{z) all have degree less than their expected degree \i = Y2j=i deggj(^). In this case we can 
obtain a shorter set of generators by substituting the delay-0 shift of the linear combination 
g"(z) for a longest generator gj(z) that is involved in the combination. 

Finally, we ask whether the set of shifts of the {g^-(z)} is catastrophic — i.e., whether there exists 
any infinite linear combination y(z) = G'(z)xi(z) of the polynomial generators g'j(z) that produces 
a finite sequence y(z). As shown in [HE], this occurs if and only if there is some polynomial p(z) € 
Ffz^ 1 ] other than z _1 and some finite fc-tuple u(z) such that g"(z) = y(z)/p(z) = G' (z)(u(z) /p(z)) 
is finite, whereas u(z)/p(z) is infinite. This occurs if and only if the matrix G'{z) mod p(z) has less 
than full rank, which occurs if and only if the k x k minors of G'(z) are all divisible by p(z). Then 
we can obtain a shorter set of generators by substituting the delay-0 shift of the linear combination 
g"(z) for a longest generator Ej(z) that is involved in the combination. 

To detect this situation, we can in principle compute the k x k minors of G'(z), and see whether 
they have any common factor p{z). It turns out that p(z) is an invariant factor of G'(z) [I], so any 
efficient algorithm for finding invariant factors of polynomial matrices may be used to find p(z). 
Then there exists some linear combination of the generators g'j(z) that equals zero modulo p(z); 
i.e., is divisible by p(z). Dividing this combination by p(z), we obtain our shorter generator g"(z). 

Example 3. Consider the multivariable LTI system C over any field F generated by the 3x2 
matrix 



G(z) 



1 1 - az- 1 
1-/3Z- 1 1 
1--/Z- 1 \-bz~ 1 



The 2x2 minors of G{z) are (a + j3)z~ 1 — a(3z~ 2 , (a+^ — 5)z~ 1 — wyz~ 2 , and (7 — (3 — 5)z~ 1 +j35z~ 2 . 
Since all have delay greater than zero (or, alternatively, since all have a common divisor p(z) = z^ 1 ), 
the set of shifts of gi(z) = (1, \—j3z~ x , 1— 7-z -1 ) and g2(z) = (l—az -1 , 1, \-5z~ 1 ) does not have the 
predictable delay property. Indeed, the linear combination gi(.z)— g2(<z) = (az' 1 , — /3z _1 , (5— 7)z~ 1 ) 
has delay 1 and length 1, and its delay-0 shift (a, — f3, (5 — 7)) (obtained by dividing out the common 
divisor p(z) = z~ l ) may replace of g\(z) or gz(z) as a shorter fundamental generator. □ 

As this development suggests, the predictable delay, predictable degree, and non-catastrophic 
properties may be seen as special cases of the "no common divisor" property, for the cases of 
p(z) = z~ l , p(z) = z, and all other polynomials, respectively; see the appendix of [6j. 

When the ground field F is the complex field C, then the minors of G'(z) have no common 
divisor p(z) E C[z _1 ] if and only if they have no common degree-1 divisor z~ l — a for any a € C. 
As Example 3 illustrates, the case a = corresponds to a test of the predictable delay property; 
by interchanging z and z~ x we can see that the case a = 00 (a -1 = 0) corresponds to a test 
of the predictable degree property. Since G'(z) mod z~ l — a is the complex matrix obtained by 
"evaluating" G'(z) at z~ l = a, this ultimately implies that G'(z) is a set of shortest fundamental 
generators for C if and only if G'(z) has full rank when evaluated at z" 1 = a for all a € C U 00. In 
system theory, this property is sometimes expressed in terms like the following: "The matrix G'(z) 
has no zeroes anywhere in the complex plane, including at zero and at infinity." 
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VI. Duality 



An alternative way of defining a linear system C is via a set of generators for its orthogonal system 
C 1 - . In linear system theory, such a representation of C is sometimes called a kernel representation, 
whereas a representation in terms of generators for C is called an image representation. 

If B 1 - is a shortest basis for C^-, then C is the set of all trajectories that are orthogonal to all 
trajectories in B^~. A fundamental duality result is that any minimal state space for has the 
same dimension as the corresponding minimal state space for C. These results lead to a minimal 
realization for C in "observer canonical form" [ lTj . We also determine the dimensions of the minimal 
transition spaces of C L . 

A. Orthogonal systems 

Each symbol alphabet Ak is a finite-dimensional vector space over F, and therefore has a dual space 
Ah of the same dimension such that for all £ A^, £ A^ there is a well-defined inner product 
(0^, Sfc) £ F. Commonly A^ is the set ¥ n of n-tuples over F; then Ak may also be taken as F n , with 
the inner product being defined in standard componentwise fashion. 

If _A = Ilfcex^fc ^ s the se t °f au Laurent trajectories, then its dual space A = Ylkei k * s ^ e 
set of all anti-Laurent trajectories (i.e., all nonzero trajectories with finite degree, plus 0). The 
inner product between a trajectory a £ A and a dual trajectory a £ A may then be defined by 
(a, a) = J2kei( a ki a k)', this sum is well defined if a is Laurent and a is anti-Laurent, because then 
only a finite number of terms in the sum are nonzero. 

The orthogonal system is then defined as set of all trajectories a £ A whose inner product 
with all trajectories a £ C is zero. C 1 - is linear, and its orthogonal system is C. Therefore C 
may be characterized as the set of all a £ A that are orthogonal to a basis B 1 - for C^. 

Example 2 (cont.). The orthogonal code C 1 - to the code C of Example 2 is C itself. Thus an 
8-tuple a £ (F2) 8 is in C if and only if it is orthogonal to the four generators gi, g2, g3, g4- □ 

If a linear system C is Laurent complete (the Laurent completion of its finite subcode), then C 1 - 
is anti-Laurent complete, so there is a nice symmetry between a system and its orthogonal systemjf] 

An LTI system C over a field F is invariant both under multiplication by elements of F and 
under time shifts, and therefore is invariant under multiplication by Laurent sequences in ¥((D)) or 
F((z -1 )), where multiplication of Laurent trajectories by Laurent sequences is defined by sequence 
convolution. Thus we usually prefer to regard an LTI system as a linear system over ¥({D)) or 
F((z -1 )). In this case the orthogonal code C 1 - is defined by an inner product in F((D)) or F((z -1 )), 
and is also linear over F((D)) or F((z -1 )). It is easy to see that the orthogonal code C 1 - so defined 
is the time reversal of the orthogonal code according to the earlier definition using an inner product 
over F. Thus we use slightly different definitions of orthogonality for LTI and for non-LTI systems^ 

Example 1 (cont.). The linear time-invariant system C of Example 1 may be regarded as the one- 
dimensional subspace of the two-dimensional vector space (F((z -1 ))) 2 that is generated by (l,g(z)). 
The orthogonal code C 1 - is then the orthogonal one-dimensional subspace, which is generated by 
(g(z), —1). In other words, a pair (u(z),y(z)) is in C if and only if u(z)g(z) — y(z) = 0. □ 

6 This symmetry can be improved by reversing the time axis of the orthogonal code. 

7 As noted in the previous footnote, it would be better conceptually if we always reversed the time axis of the 
orthogonal system C ± , but in the general case the notation then becomes cumbersome. 
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B. Dual minimal state and transition spaces 

It is well known that the minimal state spaces Efc of the orthogonal system C 1 - have the same 
dimensions as the corresponding minimal state spaces of C. This follows from: 

• The fact that T, k is isomorphic to the past- induced state space P k - (C)/C k - , or equivalently to 
the quotient space R k - (C) / R k - (C k -) , where the restriction R k ~ maps a trajectory a = {ay ■ 
k! £ 1} defined on 1 to a restricted trajectory R k -(a) = {a k /, '■ k' € k~} defined on k~ . 

• The fact that restricted systems and subsystems are duals, in the sense that R k -(C L ) = 
(Rk-iCk-)) 1 - andi? fc -((C- L )V) = (^-(C))" 1 - This follows from the fact that (R k - (&) , R k + (&)) 
is orthogonal to (R k - (b) , R k + (0)) if and only if R k -(a) is orthogonal to R k -(h). 

• The fact that if B is a subspace of A, then A is a subspace of B^, and the quotient spaces 
A/B and B ± /A ± are dual spaces, with the same dimension. Consequently R k - (C) / R k - (C k - ) ~ 
Sjt and R k -(C ± )/R k -((C- L ) k ~) ~ T, k are dual spaces, and dimS^ = dimS^. 

Is there a corresponding duality result for minimal transition spaces? A little-known fact, 
apparently first discovered by Mittelholzer [13] , and proved in a more general context in [9] , is that 
whereas T, k and T, k are dual spaces, if T k C x A k x £fc+i is a minimal transition space of C, then 
the corresponding minimal transition space of C 1 - is the orthogonal space {T k )' L to T k in the dual 
space Ti k x A k x Efc+i, where orthogonality is defined with respect to the following bilinear form: 

((&k, a k, Cfc+i), (&k,ak, <5fc+i)) = (cr k , a k ) + (a k ,a k ) — {<J k+ \,<J k+ \) ■ 

Thus the dimensions of the minimal transition spaces of C and C 1 - are in general related as follows: 

dim7fc + dim(7fc)" L = dimX/% + dim A k + dim£fc_|_i. 

The in-space of C 1 - at time k is in some sense the dual to the out-space of C at time k, and 
vice versa, as can be seen from the following relations. Since dimX^ = dimS/% and dimTfc = 
dimSfc + dim/fc = dimSfc + i + dimOfc, we have 

dim(Tfc)" 1 " = dimSfc + dim^4fc — dimO^ = dimEfc + i + dim A k — diml k . 

Thus if I k and O k are the in-space and out-space of at symbol time k, then 

dim/fc = dim^4fc — dimO^; 
dim O k = dim A k — dim I k . 

In particular, when symbols are simply elements of F so that dim^ = 1, then dimO^ = if 
dim/fc = 1 and vice versa. 

Example 1 (cont.). Let C be any single-input, single-output linear time-invariant system, whose 
shortest basis is the set of all shifts of the fundamental input-output trajectory (b(z),a(z)), whose 
delay is and whose degree is 5 = maxjdeg b(z), deg a(z)}. Then a shortest basis for the orthogonal 
system C 1 - is the set of all shifts of the fundamental input-output trajectory (0(21), —b(z)). Thus 
the minimal state spaces of C and both have dimension 5, and the minimal transition spaces of 
C and C 1 - both have dimension 5 + 1. Also, all in-spaces and out-spaces have dimension 1. Since 
dim^4fc = 2, it is easy to check all that the relations above are satisfied. □ 
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Example 2 (cont.). The binary code C of Example 2 is generated by the shortest basis {11110000, 
00111100,00001111,01011010}, and has state-space dimension profile {0,1,2,3,2,3,2,1,0} and 
transition-space dimension profile {1,2,3,3,3,3,2,1}. The orthogonal code C 1 - is the same code, 
and thus has the same profiles. One may check that the relations above are satisfied at all times; for 
example, at symbol time 0, dim/o = dim Jo = 1, dimOo = dimOo = and dim7o = dim(To)" 1 " = 1. 
Note that since dimA/% = 1 and C = C 1 - , we must have dimOfc = if dim/fc = 1, and vice versa; 
thus dimlfc = 1 for k = 0, 1, 2, 4, while dimO^ = 1 for k = 3,5, 6, 7. □ 

Example 3 (cont.). The system C of Example 3 is a two-dimensional subspace of (F((z -1 ))) 3 , and 
has a shortest basis consisting of the shifts of the two delay-0 generators g'i(z) = (a, —/3, (S—'j)) and 
§2(2) = (1 — az _1 , 1, 1 — Sz~ 1 ), which have degrees and 1, respectively. Thus the minimal state 
spaces of C have dimension 1, and its minimal transition spaces have dimension 3. The orthogonal 
system C 1 - must thus be a one-dimensional subspace of (F((z -1 ))) 3 , and must be generated by the 
shifts of a delay-0, degree-1 generator h(z). Thus its minimal state spaces have dimension 1, and 
its minimal transition spaces have dimension 2. □ 

More generally, as we have seen, a multivariable LTI system C has a polynomial n x k generator 
matrix G'(z) = {g'j(z) : 1 < j < k}, whose k x k minors have no common polynomial factors, and 
have maximum degree \i = Ylj=\ deggj(z); then the shifts of the fundamental generators gj(z) 
form a shortest basis for C. It turns out that the orthogonal LTI system has a polynomial 
n x (n — k) generator matrix H(z) = {hj(z) : 1 < j < n — k} whose (n — k) x (n — k) minors are 
the same (up to sign) as the complementary k x k minors of G'(z) [5], such that the shifts of the 
fundamental generators hj(z) form a shortest basis for C^. Since the in-spaces and out-spaces of 
C have dimension k, and those of C 1 - have dimension n — k, it follows that the dimensions of the 
minimal state spaces of C and are both equal to /u, that the dimensions of the minimal transition 
spaces of C are fi + k, and that the dimensions of the minimal transition spaces of C 1 - are [i + n — k. 

Example 3 (cont.). For the system C of Example 3, the 2x2 minors of G'(z) are a + f3 — ufiz^ 1 , 
a + 7 — 5 — a^/z~ 1 , and 7 — /3 — 5 + /35z _1 . It follows that these are also the lxl minors 
of the generator matrix H(z) of C^, up to sign; indeed, the fundamental generator of H(z) is 
h(z) = (7-/3-<5 + /3fc- 1 ,-a-7 + (5 + a7z- 1 ,a + /3-a/3z" 1 ). □ 

C. Minimal realizations in observer canonical form 

Given a shortest basis B 1 - for the orthogonal system C 1 - to a linear system C, we can give a 
straightforward state-space realization for C, sometimes called the observer canonical form |llj . 
whose state-space dimensions are minimal at all times. Indeed, this realization may be obtained 
by dualizing the controller canonical form realization of C 1 - . 

With each generator h G B^~, we associate a one-dimensional atomic "checker" as follows. 
Roughly, for an arbitrary trajectory a € A, the realization accumulates the partial sums of the 
inner product (a, h) = Y2k€[del h degh] ( a ki %) m an "accumulator." The trajectory "checks" (is 
valid) with respect to h if the final sum in the accumulator is 0. 

More precisely, if del h < deg h, then the state spaces of the atomic checker are equal to F during 
the active state interval (del h, degh], and equal to the trivial space {0} otherwisep thus the state 
space dimension is 1 during the active interval and otherwise. The sets of allowable transitions 

8 If del h = degh, then Efc = {0} for all k £ Ts, and Tk = {(0, ak, 0) : (at, hk) = 0} at symbol time k — del h = 
degh; otherwise Tk = {(0,0,0)}. 
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Tk are as given below during the active symbol interval [del h, degh] (otherwise Tk = {(0,0,0)}): 



• For k = del h, Tk = {(0, a k , a k +i = {a k ,h k )) : a k G A k }; 

• For del h < k < degh, Tk = {(ak,a k ,(Jk + i = a k + (a fc ,/i fc )) : a fc £ A k }; 

• For fc = degh, T k = {(-(a k ,hk},a k ,0) : a k G ^4fe}. 

(It is easily checked that these are the orthogonal transition spaces under the bilinear form given 
above to those defined in Section IV-D for the controller canonical form, if h is substituted for g.) 
Evidently ak = ^2k'<k( a k' > ^fe') f° r ^ ^ degh, and a trajectory a £ i has a corresponding state 
sequence 0"(a) such that (a, <r(a)) is a valid symbol-state trajectory if and only if 

Cdegh = (Ofc, hk) = — (Odogh, ^degh); 

fcg[del h,degh) 

i.e., if and only if (a, h) = 0. 

The whole realization for C then consists of the aggregate of these atomic "checkers" for all 
h G B^~. The set of all a G A that have a compatible state sequence <x(a) in all "checkers" is the 
set of all a G A that are orthogonal to all h G £>-*-, which is precisely the linear system C. The 
number of memory elements active at any state time k G Is is the number of active h G B^ at 
time k, which is the dimension of the minimal state space for C^-, which equals dimS/;. Thus 
this aggregate "observer canonical form" realization is a minimal (and linear) realization of C. 

In an observer canonical realization of a linear time-invariant system, the lengths of the dual 
generators h G are sometimes called the observability indices of C. Thus the observability 
indices of C are the controllability indices of C 1 - , and vice versa. 

VII. Conclusion 

The "shortest basis" approach describes a linear system C by a shortest basis B that transparently 
characterizes the controllability properties of C, or by a shortest basis B 1 - for C 1 - that similarly 
characterizes the observability properties of C. In particular, B and B^ lead directly to minimal 
linear realizations of C in controller and observer canonical form, respectively. 

In this paper, we have not carried this approach through to the development of actual algorithms 
for computing minimal realizations, given a description of a linear system C. When the time axis 
I is finite, then there exist straightforward algorithms for reducing a given basis B to a shortest 
basis by detecting and correcting any failure of B to have the predictable span property; see |14j . 
When the time axis is infinite, we need also to detect and correct catastrophicity. For LTI systems, 
this is essentially a matter of finding and eliminating invariant polynomial factors, as discussed in 
Section V. For linear time- varying systems, detecting catastrophicity is in general an open question, 
depending on how the system is described. 

As shown in [Sj, this approach generalizes naturally to discrete-time group systems. As shown 
in [9], it further generalizes to linear and group systems defined on cycle- free graphs, rather than 
on a standard discrete time axis, and to some extent to general graphs. Such generality suggests 
that the "shortest basis" approach is rather fundamental. 
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Appendix. The shortest basis approach for complete systems 

In this Appendix, we relax the restriction that trajectories a G A = Y\kei^ k mus t be Laurent. 
This allows us to consider uncontrollable systems C C A, which in general will have autonomous 
components. The theory is a straightforward extension of the approach developed for controllable 
systems in the main body of the paper, but now a shortest basis B may include infinite generators. 
However, we will see that the duality theory becomes less symmetrical, since the dual space A now 
consists of only the finite trajectories in Yikex^k- 

We correspondingly relax the restriction that linear combinations must be Laurent. Thus, if B 
is a basis for C, then C is the set of all linear combinations of generators in B. This implies that C 
is "complete," not just "Laurent complete." 

We continue to define a system C as controllable if it is generated by its finite trajectories, where 
"generated" is now understood in the sense of unrestricted linear combinations. A complete LTI 
system need not be controllable, as is shown by the following simple example. 

Example A (repetition system). Over any field F, the repetition system C is defined as the set of 
all bi-infinite trajectories a € ¥ z such that = a for all k € Z, for some a € F. C is evidently a 
linear time-invariant system, and indeed a one-dimensional subspace of F z that is generated by the 
bi-infinite all-one trajectory 1 G F z . Since the only finite trajectory in C is the all-zero trajectory 
0, C is uncontrollable. □ 

The controllable subsystem of a linear system C is defined as the subsystem C c generated by 
all finite trajectories of C. The uncontrollable component of C may be defined abstractly as the 
quotient space C/C c . 

As is well known, the minimal state space theorem continues to hold for uncontrollable systems, 
and it is easy to see that the minimal transition space theorem does also. Furthermore, Theorems 
4 and 5 continue to hold; i.e., the minimal state space (resp. transition space) dimension at state 
(resp. symbol) time k is the number of active generators at time k. Thus if we continue to require 
that minimal state spaces be finite-dimensional, then the dimension of C u must be finite. Thus C u 
must have a basis B u consisting of a finite set of infinite trajectories in C (e.g., the all-one trajectory 
1 in Example A). 

For the controllable subsystem C c , the "shortest basis" approach of the main body of the text 
goes through without significant change. In particular, we may still obtain a basis B c for C c by our 
greedy construction. This construction may be continued to construct a shortest basis B u for C u 
consisting of a finite set of infinite generators. Together, B c and B u form a shortest basis B for C. 

Minimal linear realizations in controller canonical form may be constructed as before, with the 
only difference being that some of the atomic generators may be active for an infinitely long time. 
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Example A (cont.). For a repetition system C, the subsystems C k - and C k + are trivial at all times 
k € Z, so the minimal state space and transition space have dimension 1 at all times. Any nonzero 
trajectory in C may be taken as the single generator in a shortest basis B; such a generator is active 
at all times. A minimal realization of C in controller canonical form is given by the one-dimensional 
atomic realization whose transition space is Tk = {(a, a, a) : a € F} at all times. □ 

It turns out that the dual space to the complete space JlfceJ^fe i s the space A consisting of 
the finite trajectories in Ilfcex^ fc - This ensures that the inner product (a, a) of a trajectory a. £ A 
and a trajectory a 6 A is well defined, since such an inner product is a sum of only finitely many 
nonzero terms (0^,0^). Linear combinations in A are restricted to finite linear combinations. 

Consequently, the orthogonal system C L C A to a complete system C C A is a finite system; 
i.e., a system all of whose trajectories are finite. Such a system is necessarily controllable — i.e., 
generated as the set of all finite linear combinations of its (finite) elements. Therefore the shortest 
basis approach of the main body of this paper applies directly to C^. 

Example A (cont.). The orthogonal system £+ to a repetition system C is the set of all finite tra- 
jectories a£F z that are orthogonal to the all-one trajectory 1; i.e., the set of all finite trajectories 
a G F z whose components sum to zero: Xlfcez ®k = 0. Note that C L is also a time-invariant system. 

Since a zero-sum trajectory cannot have length 1, the shortest nonzero trajectories in have 
length 2 and are of the form a(D k — D k+1 ) for some a £ F and k £ Z. The set of all shifts of the 
fundamental generator 1 — D generates C 1 - , and is therefore a shortest basis B for C^. □ 

Our duality results for minimal state spaces and transition spaces of C 1 - continue to hold. From 
a shortest basis for C ± , we can construct a minimal realization of C 1 - in controller canonical form, 
or a minimal realization of C in observer canonical form. 

Example A (cont.). The elements D k — D k+l of a shortest basis B for C 1 - all have length 2. One 
is active at each state time, and two are active at each symbol time. Therefore the minimal state 
space dimension is 1 at each time, the same as the minimal state space dimension for C, and the 
minimal transition space dimension is 2, which equals 

dim Sfe + dim A k + dim Sfc+i — dim 7^ = 1 + 1 + 1 — 1. 

The in-space and out-space dimensions of C are at all times, and those of C 1 - are 1 at all times. 

A minimal realization for C 1 - in controller canonical form may be constructed from an infinite 
set of one-dimensional atomic realizations, one for each k € Z, of which the A;th produces an 
output sequence a(D k — D k+l ) and is active only at one state time, namely k + 1. Similarly, a 
minimal realization for C in observer canonical form may be constructed from an infinite set of one- 
dimensional atomic checkers, one for each k € Z, of which the kth checks that the inner product of 
a with D k — D k+1 is zero, and is active only at state time k + 1. □ 

In summary, the shortest basis approach applies equally well to complete systems. 
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